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website,  we  may  discover  more  about  its  structure  and  behavior.  This  study  consists  of  a framework  for  collection  of 
testability  measures  during  the  manual  testing  process  of  a Website.  The  measures  gathered  in  this  way  can  take  account  of 
dynamic  and  content  driven  aspects  of  web  applications,  such  as  form  structure,  client-side  scripting  and  server-side  code. 
Their  goal  is  to  capture  measurements  related  to  on-going  testing  activity,  indicating  where  additional  testing  can  best  lead 
to  higher  overall  coverage.  The  current  Web  applications  are  in  continuous  evolution  to  provide  new  and  more  complex 
functionalities,  which  can  improve  the  user  experience  by  means  of  adapting  and  dynamic  changes.  Since  testing  is  the 
most  frequently  used  technique  to  evaluate  the  quality  of  software  applications  in  industry,  manual  testing  approaches  will 
be  necessary  to  evaluate  the  quality  of  future  (and  more  complex)  web  applications. 

KEYWORDS:  Systematic  Literature  Review  (SLR),  Manual  Testing,  Systematic  Mapping  (SM) 

INTRODUCTION 

The  quality  assurance  of  any  system  depends  on  its  test.  To  do  manually  testing  is  time  consuming,  expensive.  In 
Web-site  testing  the  site  is  completely  tested  before  going  production  environment.  This  could  help  to  address  the  issues  in 
web  Site  before  exposed  to  public  like  the  Functional  issues,  web  application  security,  web  services  issues,  integrations 
issues,  environment  issues  and  its  ability  to  handle  traffic  is  checked.  In  this  stage  of  Web  site  Testing  making  effort  to  find 
out  the  possible  bugs  in  the  system. 

Manually  a website  can  be  tested  using  the  steps  as  shown  in  the  figure. 
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One  of  the  challenges  of  testing  web  applications  derives  from  their  dynamic  content  and  structure.  As  we  test  a 
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Figure  1:  Manual  Testing  Lifecycle 
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NEED  FOR  THE  STUDY 

Complete  testing  of  a web-based  system  before  going  live  can  help  address  issues  before  the  system  is  revealed  to 
the  public.  Issues  such  as  the  security  of  the  web  application,  the  basic  functionality  of  the  site,  its  accessibility  to 
handicapped  users  and  fully  able  users,  its  ability  to  adapt  to  the  multitude  of  desktops,  devices,  and  operating  systems,  as 
well  as  readiness  for  expected  traffic  and  number  of  users  and  the  ability  to  survive  a massive  spike  in  user  traffic,  both  of 
which  are  related  to  load  testing. 

LITERATURE  REVIEW 

Manual  Testing  is  a process  carried  out  to  find  the  defects.  In  this  method  the  tester  plays  an  important  role  as 
end  user  and  verify  all  features  of  the  application  to  ensure  that  the  behavior  of  the  application.  The  Manual  Testing  is  very 
basic  type  of  testing  which  helps  to  find  the  bugs  in  the  application  under  test.  The  Test  Plan  is  created  & followed  by  the 
tester  to  ensure  that  the  comprehensiveness  of  testing  while  executing  the  test  cases  manually.  It  is  not  necessary  to  have 
knowledge  of  any  testing  tool  for  manual  software  testing. 

The  main  Goal  of  Manual  Testing  is  to  make  sure  that  the  application  under  test  is  defect  free  and  software 
application  is  working  as  per  the  requirement  specification  document. 

This  type  includes  the  testing  of  the  Software  manually  i.e.  without  using  any  automated  tool  or  any  script.  In  this 
type,  tester  takes  over  the  role  of  end  user  and  test  the  Software  to  identify  any  un-expected  behavior  or  bug.  There  are 
different  stages  for  Manual  Testing  like  Unit  testing.  Integration  testing.  System  testing  and  User  Acceptance  testing. 

A test  plan  document  is  created  by  test  lead  which  describes  the  detailed  and  systematic  approach  to  testing  a 
software  application.  The  test  plan  typically  includes  a complete  understanding  of  what  the  ultimate  workflow  will  be.  To 
ensure  the  completeness  of  testing  (100%  test  coverage)  test  cases  or  test  scenarios  are  created. 

DOGAN,  Serdar  M.S.,  Department  of  Information  Systems,  The  Web  has  had  a significant  impact  on  all  aspects 
of  our  society.  As  our  society  relies  more  and  more  on  the  Web,  the  dependability  of  web  applications  has  become 
increasingly  important.  To  make  these  applications  more  dependable,  for  the  past  decade  researchers  have  proposed 
various  techniques  for  testing  web  based  software  applications. 

As  this  research  area  matures  and  the  number  of  related  papers  increases,  it  is  important  to  systematically  identify, 
analyze,  and  classify  the  v publications  and  provide  an  overview  of  the  trends  and  empirical  evidence  in  this  specialized 
field.  We  systematically  review  the  body  of  knowledge  related  to  web  application  testing  through  a systematic  literature 
review  (SLR)  study.  This  SLR  is  a follow-up  and  complimentary  study  to  a recent  systematic  mapping  (SM)  study  that  has 
been  conducted  in  this  area.  As  part  of  this  study,  we  pose  three  sets  of  research  questions,  define  selection  and  exclusion 
criteria,  and  synthesize  the  empirical  evidence  in  this  area. 

Benefits  of  Website  Testing 

Before  launching  a website,  it  is  very  important  to  test  its  usability.  Usability  testing  involves  evaluating  the 
functionality  and  features  of  your  site  by  trying  it  out  with  users. 

The  following  are  some  of  the  benefits  of  website  usability  testing: 
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• Lower  costs 

• Enhanced  retention  rate 

• Enhanced  brand  image 

Your  website  is  a very  important  element  of  your  marketing  strategy  and  overall  brand  image.  If  visitors  have  a pleasant  user 
experience  on  your  site,  they  will  also  have  a favorable  impression  of  your  brand.  Otherwise  they  will  just  click  away  to  a competitor’s  site. 

Site  Phases 

The  project  is  assigned  around  4-5  months  before  release.  Then  the  site  phases  have  individual  release  dates. 
Phases  are  listed  as: 

• Mini  teaser  site 

• Teaser  Site 

• Main  Site 

• DVD  Site 

• DVD  Updates 

Each  site  phase  diff  ers  other  by  the  quantity  of  contents  and  functionality.  Let’s  see  how, 

• Mini  teaser  site:  This  site  is  released  to  provide  just  an  idea  about  upcoming  movie  by  production  house.  This 
contains  minimum  content  related  to  movie  for  highlight. 

• Teaser  Site:  Next  version  to  mini  teaser  is  teaser  site.  This  has  some  more  information  about  movie.  Some  release 
dates  and  some  more  added  features. 

• Main  Site  : Next  to  teaser  site,  site  becomes  main  site  with  lots  of  contents  and  activities  on  it.  It  adds  lots  of 
functionalities  in  backend  too.  This  phase  is  generally  1 month  before  movie  release  date. 

• DVD  Site:  This  phase  is  quite  late  from  the  actual  release  date  of  the  movie.  This  keeps  the  content  of  main  site  as 
is  and  adds  some  extra  content.  After  main  site  let’s  have  a look  on  other  marketing  sites  related  to  production 
houses.  World  wise  release  needs  a lot  to  market. 

Applications 

These  are  related  or  based  on  the  famous  character  or  famous  activity  in  the  movie.  Technologies  other  than 
HTML  are  used  to  create  those  applications.  They  may  be  some  trivia,  photo  applications,  card  creators,  games  or  FB 
applications.  They  will  need  input  from  users. 

• Portals:  Portals  are  sites  which  give  brief  information  about  upcoming  products,  or  upcoming  movies  or 
upcoming  phases  of  movie  websites  of  particular  production  house.  These  also  have  information  about  some  old 
milestone  movies  by  production  houses. 

• Games:  These  are  small  multimedia  games  based  on  famous  character  or  activity  in  the  movie. 
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These  games  are  hosted  on  so  many  games  sites  also. 

• Banners:  These  are  little  informer  of  the  movie.  They  are  multimedia  advertisement  created  to  host  on  other 
websites. 

• ITunes  tie  ups:  This  is  for  the  promotion  of  the  trailers  of  the  movie.  Trailers  are  hosted  on  ITunes  sites. 

OBJECTIVES  OF  THE  STUDY 

• To  understand  manual  testing  in  detail  and  Differentiating  the  Phase  of  the  Site 

• Logging  the  defect  report  by  Manually  Testing  the  Site 

• Comparison  of  Manual  Testing  & Automation 

• Design  of  a framework  for  collection  of  testability  measures  during  the  manual  testing  process 

RESEARCH  METHODOLOGY 

Quantitative  Research  Methodology 

The  Testing  of  the  Tumblr  Site  http://clients.homebrewagencv.com/bvs-tumblr.  The  site  is  tested  from  the  staging 
period  to  it  being  on  Live.  Various  defects  are  logged  by  opting  the  Manual  approach  for  Testing. 

The  rounds  of  QA  done  manually  shows  the  efficiency  of  manually  testing  the  Site. 

Source  and  Methods  of  Data  Collection 

The  source  for  the  data  collection  is  the  defect  Report  where  the  defects  are  logged  while  testing  the  site 
manually.  The  defect  report  consists  of  the  Bug  reported  in  the  Podio.  The  bugs  have  follow  the  lifecycle  until  they’re 
closed. 

Methods  of  Data  Analysis  and  Statistical  Techniques 

After  uncovering  a defect  (bug),  testers  generate  a formal  defect  report.  The  purpose  of  a defect  report  is  to  state 
the  problem  as  clearly  as  possible  so  that  developers  can  replicate  the  defect  easily  and  fix  it.  A defect  reporting  tool,  Podio 
is  used  and  the  elements  of  a report  are  Bug  ID,  Title,  Description,  Steps  to  be  reproduced.  Status,  Priority,  Type,  etc. 

After  logging  the  reports  one  can  generate  various  graphs  in  order  to  track  the  efficiency  of  the  work.  The 
Description  of  the  Bug  is  written  in  detail  so  the  developer  can  understand  and  reproduce  it  in  order  to  fix  it. 

Limitations  of  study 

• Manual  testing  is  a laborious  activity  that  requires  the  tester  to  possess  a certain  set  of  qualities;  To  be  patient, 
Observant,Speculative,Creative,Innovative,Open-minded, Resourceful, Un-opinionated,Skillful. 

• GUI  objects  size  difference  and  color  combination  etc  is  not  easy  to  find  out  in  manual  testing. 

• Load  testing  and  performance  testing  is  not  possible  in  manual  testing. 

• Running  test  manually  is  very  time  consuming  job. 
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• Regression  Test  cases  are  time  consuming  if  it  is  manual  testing. 


• Use  of  Automation  tools  increases  the  performance  & the  coverage  of  maximum  test  cases  is  possible  in  time 
critical  projects 

Data  Analysis  and  Interpretation 


• The  following  table  gives  the  number  of  bugs  found  out  by  manually  testing  the  tumblr  Site. 

• It  also  tracks  the  further  development  of  the  site  and  logging  more  ‘New  Bugs’. 

• The  number  of  Fixed  Bugs  are  given  in  the  last  column. 

• The  tumblr  Website  is  Manually  tested  throughout  the  entire  phase  giving  the  below  defect  report. 

• Thus,  the  rounds  of  QA  done  can  be  more  efficient  if  few  things  are  automated. 

• The  below  graph  explains  the  Life  cycle  of  the  Manual  Testing. 


Table  1:  Defect  Report 


Bugs  Founded 

New  Bugs 

Fixed  Bugs 

22 

7 

8 

29 

5 

7 

34 

2 

11 

36 

0 

10 

FINDINGS 

• We  are  able  to  log  in  all  36  bugs  in  3 rounds  of  Manual  QA. 

• The  tumblr  Site  above  is  in  the  ‘Main’  site  phase 

• Various  Bugs  are  found  such  as  Functional,  Usability,  Social,  Legal,  SEO 

• It  is  found  that  manually  testing  the  Site  is  Time  consuming  & expensive. 

• By  automating  few  things  the  coverage  of  the  Test  cases  can  be  covered  early. 

• The  defect  report  gives  some  light  on  the  Bug  Life  cycle  as  well. 
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• The  Manual  testing  Lifecycle  is  repetitive. 

DESIGN  OF  A FRAMEWORK  FOR  COLLECTION  OF  TESTABILITY  MEASURES 

Web  Application  Testing  Checklist  (Prepared  based  on  actual  Practice  in  Sycamore  Software  Solutions  Pvt.  Ltd 

Let  see  what  all  testing  is  to  be  carried  out  on  in  software  web  testing.  The  testing  is  totally  based  on  your  web 
testing  requirements  but  following  is  the  standard  checklist  of  web  testing: 

FUNCTIONALITY  TESTING 

In  Functional  testing  we  need  check  the  each  component  is  functioning  as  expected  or  not,  so  it  is  also  called  as 
“Component  Testing”.  Functional  testing  is  to  testing  the  functionality  of  the  software  application  under  test.  It  is  to  check 
the  basic  functionality  mentioned  in  the  functional  specification  document.  Also  check  whether  software  application  is 
meeting  the  user  expectations.  We  can  also  say  that  checking  the  behavior  of  the  software  application  against  test 
specification. 

In  this  testing  activities  should  include: 

• Link  Testing:  Check  for  all  broken  links  on  your  web  pages  & all  links  are  working  correctly.  Along  with  you 
can  check  the  different  links  on  you  web  pages: 

Internal  links.  Outgoing  links.  Mail  to  Links,  Anchor  Links, 

• Web  Form  testing:  In  the  web  application  testing  the  “Forms  Testing”  is  the  essential  part  of  testing  any  web 
site.  The  main  purpose  of  Form  is  to  get  the  information  from  user  & store  into  the  database.  And  keep  on  interact 
with  them.  Below  are  the  test  cases  which  should  be  considered  while  doing  form  testing: 

First  thing  to  test  in  the  form  is  the  Validations  on  each  form  fields.  Here  two  types  of  Validation  need  to  be 
consider  - “Client  side”  & “Server  side”  validations. 

Check  default  values  are  being  populated 

Check  for  all  Mandatory  fields.  Check  if  a user  not  entered  a required  field  showing  mandatory  error  message. 
Add  information  using  form  & update  information  using  form.  Tab  orders.  Check  for  the  default  values  of  fields.  Forms 
are  optimally  formatted  for  better  readability  To  optimize  web  site  in  the  search  engine  then  testing  of  HTML  and  CSS 
make  important  role.  In  this  testing  check  that  different  search  engines  can  crawl  your  site  without  any  error.  You  should 
check  for  all  Syntax  Errors,  Color  Schemas  in  the  readable,  check  for  Standard  Compliance  and  the  standards  are  followed. 

Usability  Testing 

Now  a day’s  the  Usability  testing  is  playing  an  important  role  of  any  web  application.  This  testing  is  to  be  carried 
out  by  testers  to  ensure  that  cover  all  possible  test  cases  which  targeted  audience  of  the  web  application  are  doing 
regularly. This  would  include  - Navigation  testing  of  the  web  site: 

All  possible  options  like  Menus,  Links  or  buttons  on  web  pages  should  be  visible  & accessible  from  all  the  web 
pages.  Web  pages  navigation  should  be  Easy  to  use  Help  instruction  content  should  be  clear  & should  satisfy  the  purpose. 
All  options  on  header,  footer  & left/right  navigation  should  be  consistent  throughout  the  pages. 
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Content  testing  of  the  web  site:  No  spelling  or  grammatical  errors  mistake  in  content  throughout  the  page.  Alt  text 
should  be  present  on  Images  No  broken  images.  Your  task  is  to  validate  all  for  UI  testing 

Follow  some  standard  on  content  building  on  web  page.  All  content  should  be  legible  & easy  to  understand.  Dark 
color  infuriates  the  users,  so  avoid  using  dark  colors  in  the  theme.  Proper  size  images  should  be  placed  on  web  page  All  the 
anchor  text  links  should  be  working  properly. 

Compatibility  Testing 

In  software  application  testing  the  Compatibility  testing  is  the  non-functional  part  of  testing.  It  is  ensuring  that 
how  application’s  working  in  the  supported  environments.  Customers  are  uses  web  application  on  different  Operating 
systems.  Browser  compatibility,  computing  capacity  of  Hardware  Platform,  Databases  and  Bandwidth  handling  capacity  of 
networking  hardware.  The  Compatibility  testing  is  to  make  sure  that  “Is  web  application  show  correctly  across  different 
devices?” 

This  would  include- 

• Browser  Compatibility  Test:  Web  applications  are  rendering  differently  on  different  browsers.  The  objective  of 
browser  compatibility  testing  is  to  ensure  that  no  any  errors  on  the  different  web  browsers  while  rendering  the 
sites.  In  Browser  Compatibility  Testing  you  need  to  ensure  that  your  web  application  is  being  displayed  properly 
on  different  browsers.  Also  check  AJAX,  JavaScript  and  authentication  are  functioning  correctly. 

• OS  Compatibility:  In  new  technology  newer  graphics  designs  are  used  & different  APIs  are  used  which  may  not 
work  on  different  Operating  systems.  Also  on  rendering  of  different  objects  like  text  fields,  buttons  may  display 
different  on  different  Operating  System.  So  testing  of  web  application  should  be  carried  out  on  different  OS  like 
Windows,  MAC,  Solaris,  Unix,  Linux  with  different  OS  flavors. 

• Mobile  Browsing:  In  latest  Mobile  technology  you  test  out  Mobile  Browser  Compatibility  too.  It  may  be  possible 
of  Compatibility  issues  on  Mobile  browsers.  So  in  the  new  Mobile  technology  age  you  testing  of  web  pages  on 
mobile  browsers  should  be  carried  out. 

Database  Testing 

Data  reliability  is  key  part  in  the  Database  testing.  So  for  web  application  should  be  thoroughly  tested.  Testing 
activities  would  include- 

Check  if  queries  are  executed  without  any  errors.  Creating,  updating  or  deleting  data  in  database  should  maintain 
the  data  integrity.  More  time  should  not  take  to  execute  the  queries,  if  required  tune  the  queries  for  better  performance. 
Check  load  on  database  while  executing  heavier  queries  & check  the  result.  Collect  data  from  database  & represent  on  the 
web  pages  correctly. 

Crowd  Testing 

Crowd  testing  is  when  a large  group  of  perfect  strangers  tries  your  product  then  gives  you  phenomenally 
helpful  feedback  on  usability,  bugs  and  features. 


Impact  Factor(JCC):  1.7539-  This  article  can  be  downloaded  from  www.impactjournals.us 


98 


Z.  S.  Mulla  & Pooja  Kamble 


To  test  the  software  application  Crowd  testing  can  be  used.  It  not  limited  to  web  applications,  but  for  all  kinds  of 
applications  including  mobile  application  testing.  Crowd  testing  is  dependent  on  the  quality  of  the  crowd.  Also  it  depends 
on  a crowd  that  is  composed  out  of  a large  group  of  diver’s  people.  It  used  do  system  tests  for  performance  and  usability 
testing.  Simply  this  is  complementary  to  ‘normal’  testing.  The  mainly  complicated  job  of  crowd  testing  is  determining  a 
good  enough  crowd. 

Interface  Testing 

In  the  Interface  testing  mainly  three  areas  should  be  covered:  Web  Server,  Application  Server  and  Database 
Server.  Ensure  that  all  the  communications  between  these  all  servers  should  be  carried  out  correctly.  Verify  that  if 
connection  between  any  servers  is  reset  or  lost  then  what  is  happing.  Check  if  any  request  interrupts  in-between  then  how 
application  is  responding.  On  returns  of  any  error  from  web  server  or  database  server  to  application  server  then  error 
should  be  Errors  are  handled  properly  & display  such  errors  to  the  user. 

Web  Server:  Check  if  all  web  requests  are  accepting  and  not  any  requests  are  denied  or  leakages. 

Application  Server:  Check  if  request  is  sending  correctly  to  the  any  server  & displayed  correctly.  Check  if  errors 
are  catch  properly  & displayed  to  admin  user.  Database  Server:  Check  if  database  server  is  returns  correct  result  on  query 
request.  Check  if  all  three  servers  are  connected  to  each  & test  request  is  processing  correctly.  And  any  error  in  between 
then  error  should  be  displayed  to  user. 

Performance  Testing 

Performance  testing  is  to  check  the  web  application  is  working  under  the  heavy  load.  Performance  testing  is 
categorized  into  two  parts:  Web  Stress  Testing,  Web  Load  Testing 

This  would  include- 

Check  if  response  times  of  Website  application  under  different  speeds  of  connections 

Check  if  site  handles  many  simultaneous  user  requests  at  same  time. 

Check  if  how  your  web  application  sustain  under  the  peak  loads 

Check  if  large  input  data  from  users. 

Check  the  behavior  of  web  application  if  simultaneous  connection  to  Database. 

Check  if  how  the  web  site  pulls  through  if  crash  occurs  due  to  peak  load. 

Check  if  optimization  methods  such  as  reduce  load  times  by  enabling  cache  on  browser  client  and  server  side, 
gzip  compression  etc  Check  if  any  hardware  memory  leakage  errors 


Security  Testing 

The  security  testing  is  carried  out  to  ensure  that  is  any  information  leakage  in  terms  of  encrypting  the  data.  In  e- 
commerce  website  the  Security  testing  is  play  an  important  role.  In  this  testing  check  if  secure  information  is  to  check 
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whether  how  to  store  sensitive  information  such  as  credit  cards  etc.  Testing  Activities  will  include-  Check  if  unauthorized 
access  to  secure  pages,  if  user  changes  from  “https”  to  “http”  (secure  to  non-secure)  in  secure  pages  then  proper  message 
should  be  display  and  vice  versa. 

Check  if  accessing  internal  pages  directly  entering  URLs  in  browser.  If  login  is  required  then  user  should 
redirected  to  login  page  or  appropriate  message  should  be  displayed.  Most  of  the  information  related  to  transactions,  error 
messages,  login  attempts  should  be  logged  in  log  file.  Check  if  restricted  files  are  able  to  access  for  download.  Check  if 
internal  Web  directories  or  files  are  not  accessible  unless  & until  not  configured  for  download.  Check  if  CAPTCHA  is 
added  & working  properly  for  logins  to  prevents  automates  logins  attempts.  Check  if  try  to  access  others  information  by 
changing  parameter  in  query  string.  For  example  if  you  are  editing  the  information  & in  URL  you  are  seeing  UserlD  = 123, 
try  to  change  this  parameter  values  & check  if  application  is  not  providing  the  other  users  information.  It  should  display 
Access  denied  for  this  user  to  view  others  users  information. 

Check  if  sessions  are  got  expired  after  pre -defined  amount  of  time  if  user  not  using  session. 

Check  if  user  not  able  to  pass  login  page  for  invalid  username/password  combination. 

Check  if  user  is  navigated  to  encrypted  SSL  pages  for  secure  website. 

SUGGESTIONS 

• Test  automation  may  be  able  to  reduce  or  eliminate  the  cost  of  actual  testing. 

• It  is  a onetime  investment.  Initial  cost  of  automation  testing  is  more  than  manual  testing  but  always  useful. 

• For  long  term  projects  Automation  Testing  should  be  opted  for  instead  of  manual  Testing. 

• The  automated  tests  run  quickly  & efficiently. 

• Automation  testing  is  very  much  helpful  regressions  in  testing  where  code  changes  frequently. 

• Automation  testing  can  also  be  done  on  different  machine  with  different  OS  platform  combination,  concurrently 

• Automation  runs  test  cases  significantly  faster  than  human  resources. 

CONCLUSIONS 

As  the  focus  on  automation  continues,  manual  testing  will  remain  dominant  for  the  next  few  years.  As  the  urge  to 
utilize  automation  grows  the  urge  to  uncover  the  complex  paradigm  shift  necessary  to  increase  automation  and  reduce 
manual  testing  still  needs  attention.  It  is  clear  that  a careful  and  structured  approach  is  necessary  to  increase  the  rate  of 
automation.  Since  multiple  variables  affect  automation,  a strong  foundation  followed  by  a focus  on  excellence  in  execution 
is  necessary  for  succeeding  in  automation.  Very  soon,  the  automation  of  test  execution  will  become  inevitable.  An 
organization’s  preparedness  will  dictate  its  testing  cost  structure  and  future  competitiveness. 
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